Search for: All records

Integrating spatial context into large language models (LLMs) has the potential to revolutionize human-computer interaction, particularly in wearable devices. In this work, we present a novel system architecture that incorporates spatial speech understanding into LLMs, enabling contextually aware and adaptive applications for wearable technologies. Our approach leverages microstructure-based spatial sensing to extract precise Direction of Arrival (DoA) information using a monaural microphone. To address the lack of existing dataset for microstructure-assisted speech recordings, we synthetically create a dataset called OmniTalk by using the LibriSpeech dataset. This spatial information is fused with linguistic embeddings from OpenAI’s Whisper model, allowing each modality to learn complementary contextual representations. The fused embeddings are aligned with the input space of LLaMA-3.2 3B model and fine-tuned with lightweight adaptation technique LoRA to optimize for on-device processing. 

Localization of networked nodes is an essential problem in emerging applications, including first-responder navigation, automated manufacturing lines, vehicular and drone navigation, asset tracking, Internet of Things, and 5G communication networks. In this paper, we present Locate3D, a novel system for peer-to-peer node localization and orientation estimation in large networks. Unlike traditional range-only methods, Locate3D introduces angle-of-arrival (AoA) data as an added network topology constraint. The system solves three key challenges: it uses angles to reduce the number of measurements required by 4X and jointly uses range and angle data for location estimation. We develop a spanning-tree approach for fast location updates, and to ensure the output graphs are rigid and uniquely realizable, even in occluded or weakly connected areas. Locate3D cuts down latency by up to 75% without compromising accuracy, surpassing standard range-only solutions. It has a 0.86 meter median localization error for building-scale multi-floor networks (32 nodes, 0 anchors) and 12.09 meters for large-scale networks (100,000 nodes, 15 anchors). 

This demonstration presents LiTEfoot, an ultra-low power localization system leveraging ambient cellular signals. To address the limitations of traditional GPS-based tracking systems in terms of power consumption and latency, LiTEfoot employs a non-linear transformation of the cellular spectrum to achieve efficient self-localization. Our design uses a simple envelope detector to realize spectrum folding, enabling the identification of multiple active base stations. 

In this paper, we introduce a low-power wide-area cellular localization system, called LiTEfoot. The core architecture of the radio carefully applies non-linear transform of the entire cellular spectrum to obtain a systematic superimposition of the synchronization signals at the baseband. The system develops methods to simultaneously identify all the base stations that are active at any cellular band from the transformed signal. The radio front end uses a simple envelop detector to realize the non-linear transformation. We build on this low-power radio to implement a self-localization system leveraging ambient 4G-LTE signals. We show that the core system can also be extended to other cellular technologies like 5G-NR and NB-IoT. The prototype achieves a median localization error of 22 meters in urban areas and 50 meters in rural areas. It can sense a 3GHz wideband LTE spectrum in 10ms using non-linear intermodulation while consuming 0.9 mJ of energy for a PCB-based implementation and 40 𝜇J for CMOS simulation. In other words, LiTEfoot tags can last for 11 years on a coin cell while continuously estimating location every 5 seconds. We believe that LiTEfoot will have widespread implications in city-scale asset tracking and other location-based services. The radio architecture can be useful beyond low-power self-localization and can find application in synchronization and communication on battery-less platforms. 

This paper presents LiTEfoot, an ultra-low power, wide-area localization system leveraging ambient cellular signals to address the limitations of traditional self-localization systems in terms of power consumption and latency. LiTEfoot uses a non-linear transformation of the cellular synchronization signal to efficiently achieve self-localization by systematically superimposing signals at the baseband. A simple envelope detector is used to realize this non-linear transformation, enabling the identification of multiple active base stations across any cellular band. The system is designed to operate with low power, consuming only 40 𝜇Joules of energy per localization update, achieving a median localization error of 22 meters in urban areas.